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Impact of Student Educational Outcomes Assessnent on Virginia 

Institutions of Higher Education 

Bernard H. Levin* and Darrel A. Clowes'-' 

Institutions of higher education seen vulnerable to forces 
exerted by the larger society. For e^cample, during the late 1960s 
and early 1970s we emphasized behavioral objectives both in 
administration and instruction. Then we experienced frenzies of 
"evaluation of teaching." in recent years, "institutional 
effectiveness" has become a byword in higher education [e.g.. 
Southern Association of Colleges and Schools, 1988]. No operational 
definition of institutional effectiveness has been universally 
adopted. In many institutions it has translated [often via state 
mandate] to student Educational outcomes Assessment [SEOA]. 

Two broad questions are worthy of examination: 

1. Does institutional effectiveness affect higher 
education institutions differentially? 

2. What is the effect of SEOA on administration, faculty, 
academic programs, and governance structures within 
institutions of higher education and what is the effect 
upon external agencies and institutions? 

Institutional effectiveness ir. an ill defined concept, welker 
[1990] has traced the literature on institutional effectiveness and 
higher education from 1970 to 1990. He found it to peak in the late 
1970 's for all of higher education and thereafter to gradually 
decline. All sectors of higher education are represented similarly 
in the literature until the late 1980 's when community college 
literature appears disproportionally involved with considerations 
of institutional effectiveness. 

WelXer found that the literature on effectiveness has almost 
no basis in e*dpirical research; it is predominantly a descriptive 
and prescriptive literature and is, in our view, more properly 
labelled a literature of management efficiency. Welker 's work also 
implies that institutional effectiveness is more a concern in the 
community colleges than in other sectors of higher education. 

Student educational outcomes assessment is a logical extension 
of the institutional effectiveness movement, in fact, the concepts 
are mingled in the Southern Association of Colleges and schools 
literature, welker [1990] also found a strong inter-relationship. 
We would expect SEOA to follow the pattern of the institutional 
effectiveness movement and impact most heavily upon the community 
college. The community college has the least secure place within 
higher education [Clowes & Levin, 1989] and is the most vulnerable 
to external pressures asserted through state coordinating and 
controlling agencies and state legislatures. 

Matlick [1990] surveyed selected faculty and administrators in 
Maryland's public two and four year colleges to determine their 
attitudes toward and perceptions of SEOA's impact, she reported 
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systematic differences in attitude and perceived impact by 
institutional type. Both two and four year personnel were generally 
negative toward SEOA and its effects, but two year college 
administrators and faculty were nore positive about SEOA generally 
and about its impact than were their four year counterparts. 

Not all researchers agree. Ewell & Boyer [1988] studied the 
effects of state assessment mandates in five states, including 
Virginia, and reported that "[C]ontrary to e>:pectation, we found no 
substantial correlation between type of institution and response to 
state assessment mandates." The present study provides additional 
information on this issue. 

Ewell [1985] proposed that the products of SEOA must be used 
if the process is to make a difference. Little empirical evidence 
has been published on the effect of SEOA within institutions of 
higher education and external to higher education. Evidence of 
use within the institution would be improved student learning and 
the redirection of administrative and faculty attention toward 
teaching and learning. Evidence of use without the institution 
would be enhanced credibility among Key constituencies and the 
general public. 

Geiger [1990] has conducted case studies of community colleges 
in Virginia. He found evidence of procedural changes in the 
administrative support structure of institutions but no reported 
changes in student learning, curriculum, or faculty involvement in 
the governance stiuctures. The effect on external constituencies 
is also little studied. Adelman [1989] suggests that state 
coordinating bodies and legislatures want assurance, or 
reassurance, about the quality of undergraduate education. Aper 
[1989] surveyed influential individuals in Virginia and concluded 
that the political goal of SEOA was reassurance, not demonstrated 
proof of excellence. 

Virginia institutions have been under a state mandate for 
SEOA for about four years. While the Virginia mandate is not nearly 
as rigid as, for example, that of Florida [Losak, 1988], SEOA in 
Virginia is clearly of the "name your own poison" variety — each 
institution has some, although only some, maneuvering room. 

At its inception the Virginia model, like some others, 
empb-^sized the importance of faculty involvement in SEOA design, 
data collection and interpretation, program change, and 
institutional governance [e.g.. Cress, 1988? Levin, Lazorack & 
Sears, 1988; Lumsden, 1988; Wyles, 1988]. 

While Virginia institutions have, for several years, been 
submitting reports on SEOA activities to the State Council of 
Higher Education for Virginia [SCHEV], SCHEV has emphasized data 
collection more than faculty involvement. As a result, it unclear 
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how involved Virginia faculties are. It is clear, however, that 
unless faculties are heavily involved, SEOA is unlikely to prove 
either useful or tenacious. 

The purpose of the present study was to provide some 
preliminary indications of how SEOA has developed within Virginia's 
institutions of higher education. 

In April of 1990 a survey was mailed to the Faculty Senate 
Chairman, Curriculum Committee Chairman, and Chief Assessment 
Officer of 23 community colleges, 15 public senior institutions, 
and 28 private senior institutions in Virginia [ergo, H « 66 x 3 » 
198]. These three positions were selected because their holders 
should be the most knowledgeable about SEOA, if it were operating 
as the Virginia model suggests. The survey asked three questions: 

1. Approximately what percentage of your full-time 
faculty already has been directly involved in development 
and implementation of student educational outcomes 
assessment? 

2. Has the governance structure at your institution 
changed at all in anticipation of or as a result of 
student educational outcomes assessment? 

3. In your opinion, has assessment become integrated 
into, added on to, or had no relationship to normal 
academic practices at your institution? 

After the sacond and third questions, there was space for 
comment. Many respondents chose to comment, and what they had to 
say was illuminating. Several representatives of private 
institutions [including a Dean and a staff member acting at the 
behest of her President] called us to inquire what SEOA was, and 
then assured us that they would investigate forthwith. 

The overall written response rate was 39 percent. As we 
expected, the response rate from community colleges was much higher 
than that of either group of senior institutions [62% vs. 26%, 
27%]. Also the response rate of faculty senate chairs was lower 
than either curriculum committee chairmen or assessment officers 
[26% vs. 44%, 47%]. We expected, and the data sfm to confirm, that 
faculty senate chairs were generally not knowledgeable about, or 
involved .fn, SEOA. 

The joint impact of senior institution status and faculty 
senate chair addressee on response rate is overwhelming — of 15 
surveys mailed to faculty senate chairs at Virginia's senior public 
institutions, not one was returned- This contrasts with a 52% 
response rate from community college faculty senate chairs, and an 
82% response rate from community college chief assessment officers. 
Clearly this comports well with the notion of a greater [primarily 
administrative] interest in SEOA in the community colleges. 
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This differential response rate is not attributable to 
institutional size or complexity. By far the largest institution of 
higher education in Virginia is a community college with five major 
campuses and a complex tapestry of programs, services, and 
governance structures [according to the 1990 Chrgnifflg 91 H i qngr 
Fci»ratton Almanac it had the 24th largest enrollment of all 
colleges in tie U.S. -1988 data] — and a relatively well 
developed SEOA program about which it communicates freely. In 
contrast, one of the smallest institutions of higher education in 
Virginia is also a community college. 

The responses themselves are instructive. Mean estimates of 
faculty involvement were 38% in public senior institutions, 45% in 
private senior institutions and 51% in community colleges. The 
variability was unusually high, with standard deviations in the 30s 
and 40s. The data demonstrate tremendous variability within 
institutional and respondent categories. This is likely due to lack 
of familiarity with SEOA [consistent with written and oral 
comments], although ambiguity in the wording of the question cannot 
be ruled out. 

Overall, only 15 of 74 respondents said there had been 
structural change. The written comments indicate that the most 
frequent change was to assign administrative staff to do SEOA. 
Clearly, this does not indicate that much fundamental change m 
governance has occurred, nor that faculty are heavily engaged. 

Regarding integration with normal academic processes, a 
somewhat less negative picture emerges. Of the public senior, 
private senior, and public community colleges, respectively, 57%, 
43%, and 51% believed SEOA was integrated, rather than an add-on. 
The rosiness fades, however, when one considers the oral and 
written comments. SEOA is seen by many as a mandated 
administrative activity rather than as a process for improving 
student outcomes. 

The survey data show that: , 

1. community college representatives were more likely to 
respond; faculty senate chairmen were least likely to respond; 

2. few respondents [20%] indicated that any structural change 
had occurred secondary to SEOA; even those changes were largely 
administrative in nature; ^ . 

3. most written comments, regardless of institution type or 
respondent's role, were consistent with SEOA as an administrative 
process. Even though this issue was not directly raised by the 
survey questions, we are surprised that not a single respondent 
commented that SEOA had led to the improvement of any educational 
outcome or that SEOA had led to any other positive outcome other 

than compliance; and 

4. there is little evidence of faculty leadership m SEOA. 
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our tentative conclusions, based on the survey data, are 

1. across college types and respondent types, SEOA is 
primarily perceived as an administrative process; 

2. community colleges seem most vulnerable to SEOA, as they 
have been to previous external mandates; and ,.<.k^^ 

3. SEOA in Virginia has yet to prove itself as anything other 
than yet one more reporting process. 

SEOA, like its parent institutional effectiveness, appears 
most clearly as an administrative response to «^J^®f"fi ."^jf^^^: 
This is consistent with Welker's conclusion that institutional 
effectiveness is really management efficiency under a new heading, 
we conclude that SEOA also functions as management efficiency. 

Both Matlick's [1990] data and our own demonstrate 
differential responses to SEOA by institutional \ , 
the community college is the higher education ^"8^^^"?^^°" 
vulnerable to external pressures, ^^^^t^Jf^^f^-^f t.|if°^ 

differential institutional responses. Ewell & Boyer [1988] relied 
on interviews and conducted their work two y««^s^®S^i}?^i «® 
the survey methods and the time of this study and Matlick's yield 
better evidence on this point. 

Although more research remains to be done, the ay^a^iJ^i® J^^* 
[Geiger, 1990; Matlick, 1990; and our own data] indicate that there 
is little effect upon student learning, academic programs, or 
faculty. Matlick does indicate that administrators are more 
positive about SEOA than are faculty; Aper [1989] ^^^f 
external constituencies desired reassurance rather than proof of 
effectiveness. 

We conclude that while SEOA affects community colleges the 
most, and while little change is visible from a faculty viewpoint, 
ther4 has been some administrative change. R^P*^'^^^ ^^"f "^®f 
and administrative procedures are in place so that external 
agencies are reassured and the political agenda for •='EOA is 
fulfilled, we do not know whether there are any educational gains 
from the SEOA initiative. It is possible that some institutions 
will use the initiative as an opportunity for self-examinat<on and 
improvement; however, we do not think this is an inevitable or even 
likely outcome for most institutions. 

We have observed that the behavioral model, teaching 
evaluation, and SEOA all became far more prominent [both as issues 
and as processes] in the community colleges ^h^" ^f" ^®n;°^ 
institutions; there appears to be an inverse relationship between 
institutional status and vulnerability to external forces. In 
addition, the behavioral model and evaluation of i^.^^^^wo^f" ^f''?; 
fot the most part, sunk beneath the higher education horizon, it 
remains to be seen whether SEOA will follow them. 
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